Improving TTS quality using pitch contour information of source speaker in S2ST framework

Intonation is one of the most important components of human spoken communication. Therefore, a correct prosody in a text-to-speech system contributes to a better quality in terms of intelligibility, naturalness and pleasantness. In order to achieve it, we need to obtain paralinguistic information from other information sources to complement the information in the text because natural language understanding algorithms have limitations that do not allow to interpret the meaning of a text nor have an opinion. In this paper we propose the use of the intonation of the speaker of the source language to extract semantic information in the framework of speech-to-speech translation. It allows to improve the quality of the intonation of the target language. The proposed approach tries to find intonation patterns that correlate between languages. In this way, given a pitch movement in the source language we may infer the pitch movement of the target language. It is possible to use this additional information source as input feature for the intonation model of the textto-speech module of the speech-to-speech translation system. Experimental results support the proposed approach with an increase in the MOS of evaluators (3.1→3.6 and 3.3→3.7 in a five point scale in the task of translation between Catalan and Spanish).